UNCLASSIFIED 


Defense  Technical  Information  Center 
Compilation  Part  Notice 

ADPO 12002 

TITLE:  Dependence  Structure  of  Random  Wavelet  Coefficients  in  Terms  of 
Cumulants 

DISTRIBUTION:  Approved  for  public  release,  distribution  unlimited 


This  paper  is  part  of  the  following  report: 

TITLE:  International  Conference  on  Curves  and  Surfaces  [4th],  Saint-Malo, 
France,  1-7  July  1999.  Proceedings,  Volume  2.  Curve  and  Surface  Fitting 

To  order  the  complete  compilation  report,  use:  ADA399401 

The  component  part  is  provided  here  to  allow  users  access  to  individually  authored  sections 
of  proceedings,  annals,  symposia,  etc.  However,  the  component  should  be  considered  within 
the  context  of  the  overall  compilation  report  and  not  as  a stand-alone  technical  report. 

The  following  component  part  numbers  comprise  the  compilation  report: 

ADPO  11967  thru  ADPO  12009 


UNCLASSIFIED 


Dependence  Structure  of  Random  Wavelet 
Coefficients  in  Terms  of  Cumulants 


Philippe  Naveau,  Peter  Brockwell,  and  Doug  Nychka 


Abstract.  When  the  Gaussian  assumption  for  a times  series  no  longer 
holds,  second  order  moment  properties  such  as  the  covariance  and  the 
spectrum  are  not  necessarily  sufficient  to  describe  the  dependence  struc- 
ture. Although  wavelet  models  have  been  proposed  to  de-correlate  the 
signal,  this  strategy  must  be  reexamined  when  applied  to  non-Gaussian 
processes.  The  process  of  interest  is  a continuous  parameter,  mean-squared 
continuous  real-valued  process  that  is  not  necessarily  Gaussian  or  linear. 
To  study  the  departures  from  linearity  and  Gaussianity,  we  consider  joint 
cumulants,  which  are  linear  combinations  of  higher  order  moments,  and 
their  associated  spectra.  A specific  objective  is  to  obtain  new  expressions 
for  cumulants  of  the  random  discrete  wavelet  coefficients  instead  of  the 
second  order  moments,  and  to  study  their  higher  order  polyspectra.  Con- 
ditions on  the  polyspectrum  to  give  null  wavelet  cumulants  within  and 
across  wavelet  coefficient  levels  are  derived.  Expressions  of  the  original 
cumulants  as  a function  of  the  wavelets  cumulants  are  also  given. 


§1.  Introduction 

The  covariance  and  spectral  properties  of  the  discrete  wavelet  coefficients  for 
random  continuous  real-valued  processes  have  been  extensively  studied  in 
the  past.  Among  others,  Donoho  et  al.  [4],  Flandrin  [6],  Mallat  et  al.  [13], 
Masry  [14],  and  Walter  [17]  have  investigated  the  correlation  within  and 
across  wavelet  coefficients.  Focusing  exclusively  on  the  second  order  prop- 
erties of  wavelet  coefficients  for  a Gaussian  process  is  a reasonable  task  since 
the  dependence  structure  of  Gaussian  processes  is  entirely  characterized  by 
the  covariance.  When  the  normality  assumption  no  longer  holds,  higher  order 
cumulants  are  necessary. 

Exploring  some  of  the  links  that  exist  between  wavelets  and  cumulants 
is  fairly  new.  Brillinger  [1]  studied  a non-parametric  regression  problem  with 
cumulants  and  wavelets.  In  geophysics  and  astrophysics,  Lazear  [11]  and 
Ferreira  et  al.  [5]  applied  wavelets  and  cumulants  to  seismic  data  sets  and 
to  the  Cosmic  Microwave  Background  problem.  The  dependence  structure 
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between  wavelet  packets  and  in  particular  wavelet  coefficients  has  also  been 
studied  by  D.  Leporini  and  J.C.  Pesquet  [12].  In  this  paper,  we  derive  new  and 
general  results  about  the  dependence  structure  of  random  wavelet  coefficients 
via  its  cumulants. 

The  process  of  interest  X(t),  indexed  by  the  real  parameter  t,  is  supposed 
to  be  a continuous  real-valued  process  that  is  mean-square  continuous,  and 
such  that  moments  of  some  order  l > 2 exist,  i.e. , 

sup  E\X(t)\l  < oo  and  lim  E\X(t  + h)  — X(t)\2  = 0.  (1) 

t h->0 

§2.  Cumulant  Definition  and  Properties 

If  some  useful  information  of  the  signal  is  not  contained  in  the  second-order 
covariances  (and  the  second  order  spectra),  then  one  can  still  calculate  some 
meaningful  linear  combination  of  higher  order  moments,  called  cumulants. 
Some  early  work  on  higher  order  cumulants  and  their  Fourier  transform  was 
proposed  by  Hasselman  et  al.  [8]  for  investigating  nonlinear  interaction  of 
ocean  waves,  and  Godfrey  [7]  used  it  for  the  analysis  of  economic  time  se- 
ries. Rosenblatt  with  Lii  and  Van  Atta  in  a series  of  papers  have  described 
how  higher  cumulants  could  be  used  to  study  nonlinear  transfer  of  energy  in 
turbulence. 

The  mth  joint  cumulant  of  the  set  of  random  variables  {X(t  i), ...,  X(tm)}, 
denoted  by  CUM(X(ti),  ...,X(tm)),  with  m < l,  is  given  by 

CUM(X(ti),  ...,X(tm))  = £(-l)p(p-  1)!(E  n X(tr))...(E  IJ  *(<r)), 

rgv  i re  t)p 

where  the  summation  extends  over  all  partitions  {^i, ...,  vp]  of  {1, ... ,m } with 
p = l,...,m.  From  this  definition,  we  can  notice  that  the  information  con- 
tained in  the  first  m cumulants  is  exactly  the  same  as  that  contained  in  the 
first  m moments.  However,  cumulants  have  some  advantages  over  moments. 
For  example,  cumulants  have  useful  linear  properties, 

CUM(Z  + X(h), ...,  X(tm))  = CUM(Z, ...,  X{tm))  + CUM(X(ti),  ...,X(tm)), 
CUM(aX(t1),...,X(tm))  = aCUM(X(tl),...,X(tm)), 

for  any  real  a.  Another  important  property  of  cumulants  concerns  the  depen- 
dence structure  of  the  process:  if  some  subset  of  (A'(t1), ...,  X{tm)}  is  inde- 
pendent of  the  remainder,  then  CUM(X(ti),  ...,X(tm))  is  identically  equal  to 
zero.  Hence,  the  cumulant,  CUM(X(ti), X(tm))  can  be  interpreted  as  a 
measure  of  dependence  of  {A(ti), ...,  A(fm)}.  For  the  special  case  of  Gaussian 
processes,  cumulants  of  order  higher  than  two  are  zero. 

In  the  remainder  of  this  section,  we  suppose  that  the  process  {Ai(t)}  is 
stationary  up  to  order  l,  i.e., 


E{X{t0)X{t1)...X{tl))  = E(X(t0  + + h)...X(ti  + h)),  V/i. 
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Stationarity  as  just  defined  is  frequently  referred  to  in  the  literature  as  weak 
stationarity.  For  us  however  the  term  stationarity,  without  further  qualifica- 
tion, will  always  refer  to  the  above  equality. 

Under  stationarity,  CUM(X(t),X(t  + Si),  ...,X(t  + sm_i))  does  not  de- 
pend on  t,  and  can  be  denoted  by  7m(si, ..., sm_i).  With  these  notations,  the 
second  order  cumulant  71  (u)  is  just  the  covariance  function.  The  third  order 
cumulant  72(14,  v)  is  the  same  as  the  third-order  central  moment, 

E((X(t)  - p)(X(t  + u)~  p)(X(t  + v)~  fi)), 

where  p is  the  mean  value  of  the  process. 

From  the  covariance  function,  one  may  define  the  power  spectrum,  i.e., 
the  Fourier  transform  of  71(f),  fi{a>)  = f 71  (t)  exp(— iojt)dt.  A natural  exten- 
sion is  the  mth- order  polyspectrum  defined  by 

m— 1 

7m(^li  tm  — l)  ^Xp(  i ^ ^ 0)jtj')dt\...dtrn—\, 

3 = 1 

assuming  that  the  above  Fourier  transforms  exist.  An  important  property  of 
the  polyspectra  is  that  all  polyspectra  of  higher  order  than  second  order  vanish 
when  (A(t)}  is  a Gaussian  process.  Another  characteristic  of  the  polyspectra 
is  that  the  ratio 

|/2K,^2)|2 

/l(^l)/l(^2)/l(^l  +^2) 

is  constant  whenever  the  process  {.X’(t)}  is  linear.  Hence,  the  simplest  higher 
order  spectrum,  called  bispectrum,  can  be  regarded  as  deviation  measures  from 
Gaussianity  and  linearity.  Different  statistical  tests  have  been  derived  from  it 
(see  Subbua  Rao  and  Gabr  [9]  and  Hinich  [16]). 


§3.  Random  Wavelet  Coefficients 

Consider  a discrete  orthonormal  wavelet  decomposition  of  a stochastic  process 
(X(t)}  that  satisfies  condition  (1).  The  corresponding  wavelet  coefficients 


Wj>k  = J X(t)i3jtk(t)dt 


(2) 


are  random  variables.  Here  the  equality  sign  is  to  be  understood  in  the  mean- 
square  sense,  and  ipj,k(t)  = 2^2tp(2H  — k ) is  an  orthonormal  wavelet  basis 
function  with  the  mother  wavelet  if.  A rigorous  framework  concerning  the 
construction  for  wavelet  orthonormal  basis  can  be  found  in  Meyer  [15]  and 
Daubechies  [3].  There  exist  many  candidates  for  the  mother  wavelet.  The 
simplest  example  of  an  orthonormal  wavelet  basis  is  provided  by  the  Haar 
system  for  which 


i>{x) 


W 


l 0, 


if  0 < x < 0.5, 
if  0.5  < x < 1, 
otherwise. 
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Our  main  interest  is  to  understand  the  dependence  structure  between  the 
different  wavelet  coefficients  defined  by  (2). 

In  this  article,  we  will  either  use  the  Haar  system  as  a simple  example 
of  compactly  support  wavelets,  or  a particular  type  of  band-limited  wavelets 
called  Meyer-type  wavelets  (see  Walter  [17],  Zayed  and  Walter  [18]).  Having 
a compact  support  in  the  frequency  domain  can  facilitate  the  computation  of 
wavelet  cumulants.  These  Meyer-type  wavelets  have  some  additional  attrac- 
tive features,  such  as  being  highly  smooth  (they  can  be  made  C°°)  and  having 
fast  decay  in  the  time  domain.  They  are  introduced  as  follows. 

Let  F be  any  probability  measure  supported  on  [ — e,  e]  for  some  c < n/3. 
Then  the  mother  wavelet  ip(-)  is  defined  by  its  Fourier  transform 

■0(w)  = exp(— iu)/2)[  f dF]1!2 . (3) 

J |cj/2|  — 7 r 

From  this  definition,  it  is  possible  to  check  that  the  orthogonality  and  dila- 
tion conditions  are  satisfied  for  the  wavelet  basis  generated  from  this  mother 
wavelet.  There  is  a large  class  of  distributions  that  can  be  chosen  in  equality 
(3),  and  the  edges  of  the  support,  [— e,e],  can  be  made  highly  smooth. 

§4.  Dependence  Structure 

The  dependence  structure  between  wavelet  coefficients  is  closely  related  to  the 
dependence  inside  the  original  signal.  Hence,  our  first  problem  is  to  explain 
how  to  obtain  the  joint  cumulants  of  the  wavelet  coefficients  from  the  joint 
cumulants  of  the  process.  The  first  proposition  takes  care  of  this  problem. 

Because  of  space  limitations,  the  proof  of  our  propositions  will  not  be  in- 
cluded in  this  paper.  However,  complete  details  of  the  proofs  can  be  requested 
from  the  authors. 

Proposition  1.  Let  {A'(t)}  be  a stochastic  process  that  satisfies  condition 
(1).  Suppose  that  the  joint  cumulants  of  order  m < l of  (X(f)}  exist.  Then 

m 

CUM(X(t1), ...,  X(tm))  n 1>in,kn  ( tn)dtn . 

n— 1 

Proposition  1 is  directly  applicable  to  compactly  supported  wavelets, 
since  the  product 

m 

Jl  (^n) 

n=l 

is  null  except  at  the  intersection  of  the  translated  and  dilated  supports.  For 
example,  suppose  that  the  wavelet  basis  corresponds  to  the  Haar  system.  The 
expression  of  cumulants  of  the  wavelet  coefficients  becomes 

CUM(Wjlkl,...,WjmkJ  = jr(- 1)'2*/2  / CUM(X(ti),  ...,X(tm))dtn.  (4) 

(=i  Ja> 


CUM(Wjlkl,...,Wjmkm)  = J ...j 
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The  Ai  are  rectangular  boxes  defined  by  the  tensor  product 

= (££)  In,c„  i) 

n=l,..,m 


where  the  vectors  c;  = (cu, , ...,  cm{)  represent  the  set  of  all  2m  possible  con- 
figurations of  {0,  l/2}m,  and  the  interval  In,cnl  is  defined  by 

In,cnl  = {tn  ■ 2 ~jn(kn  + cni ) <tm<  2 ~3n{kn  + cni  + 1/2),  for  n = 1, 


In  order  to  apply  (4)  to  a more  specific  example,  we  suppose  that  the  process 
{.X'(f)}  is  a zero- mean  stationary  process  with  standard  deviation  cr  and  covari- 
ance 7i (h)  = 6exp(— a|/i|),  and  ^\(h\,h2)  = eexp(— a\hi  +h 2 1),  where  a,b,c 
are  constants  that  depends  on  the  second  and  third  moment  and  other  param- 
eters describing  the  original  process.  Processes  with  such  a cumulant  function 
correspond  to  Continuous  Auto- Regressive  processes  (CAR)  (see  Brockwell 
[2])  or  equivalently  solutions  of  particular  stochastic  differential  equations  with 
non-necessarly  Gaussian  noise.  After  some  algebra,  wavelet  cumulants  sim- 
plify to 


CUM(Wjlkl,Whk,)  = - 


bKa(ji,ki) 

a2Ka(j2,k2) 


i E 

(ui,ti2)e{— i,i}2 


Hh{uia)Hj2{u2a)], 


for  2 32  (k2  + 1)  < 2 31  ki  and 


CUM(Wjlkl,Wj2k2,Wj3k3) 


cK2a(ji,ki) 

2a3Ka(j2,k2)Ka{j3,k3) 


X [ E Hn  (uia)Hh  {u2a)Hj3  («3a)] 

(ui,«2i«3)€{-1,1}3 


for  2~j3(k3  + 1)  < min(2-:,1fci,2_',2A:2)  with  Ka(j,k)  = exp(a(2_J(fc  + 0.5))) 
and  Hj(a)  = 1 — exp(a2_-,_1).  The  previous  formulas  can  be  easily  extended 
to  higher  dimensions,  and  can  be  used  to  derive  asymptotic  behavior,  e.g 
lii  — j2\  T 00  and  so  on. 

Another  possible  application  of  Proposition  1 is  to  non-stationary  pro- 
cesses. A large  variety  of  models,  such  as  the  bilinear  model,  autoregressive 
models  with  random  coefficients,  and  the  threshold  model,  have  been  pro- 
posed to  take  into  account  of  the  non-stationarity.  To  illustrate  the  use  of 
cumulants,  we  restrict  attention  to  piecewise  stationary  processes,  i.e.  a sum 
of  independent  stationary  processes: 


X(t)  = Y,l(ul<t  < ui+i)X^l\t),  where 
(=1 


at  £ a 

otherwise, 


and  X"\t)  are  independent  stationary  processes  and  the  change-points  are 
equal  to  —00  = uq  < ui  < ■ ■ • < uT  < ur+ 1 = 00.  Because  of  linear  properties 
of  the  cumulants,  we  have  immediately  that 

r m 

CUM(X(t1),...,X(tm))  = eii  < ul+1)CUM(X^(t)), 

1=1  71=1 
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where  CUM{X{l\t))  = CUM(X<-l'>(t1),...,XV'>(tm)).  Using  Proposition  1,  it 
follows  that 


r - m 

CUM( Wjk)  = £ / CUM(X«(0)  n T(u/  — tjl  V'l  + l')'4*jn,kn(.tn')dtn 

(=1  ^ n=l 

with  CUM(Wjl)  = CUM(Wj1k1,  The  above  expression  shows 

that  wavelet  cumulants  for  piecewise  stationary  processes  can  be  easily  com- 
puted for  compactly  supported  wavelets  such  as  the  Haar  system,  and  when 
each  process  X®  has  simple  cumulant  functions  (e.g.  the  CARMA  process). 

From  the  CAR  example,  we  saw  that  wavelet  cumulants  are  computable 
for  the  Haar  system,  but  the  resulting  formula  are  not  so  easy  to  manipu- 
late. Another  approach  is  to  use  band-limited  wavelets.  Simpler  expression 
of  the  wavelet  cumulants  can  be  derived.  To  illustrate  this  point,  we  look 
at  the  Meyer-type  wavelet  in  the  next  proposition.  In  this  case,  the  Meyer- 
type  wavelet  gives  null  or  small  wavelet  cumulants  within  and  across  wavelet 
coefficient  levels  under  simple  conditions. 

Proposition  2.  Let  {A'(t)}  be  a stationary  process  that  satisfies  condition 
(1).  Suppose  that  its  rnth -order  polyspectrum  fm  is  well  defined,  and  the 
orthonormal  basis  {ipjk}  is  generated  by  a mother  Meyer-type  wavelet.  If 
there  exists  some  integer  j*  in  {ji,  ...,jm}  such  that  Yljjtj-  2J  < 2J  ~2,  then 
CUM(Wjlk1, ...,  Wjmkm)  = 0.  In  addition,  if  /m(w)  and  V>(w)  are  both  in  Cp, 
then  the  cumulant  at  a fixed  resolution  level  satisfies 

CUM(Wjlkl,...,Whkm ) = 0(max  J]  \kr  - ks\~p). 

s^r 


Proposition  2 shows  that  the  Meyer-type  wavelet  transform  not  only  can 
remove  the  correlation  inside  the  original  signal,  but  in  addition  the  higher- 
order  cumulants  are  either  null  at  distant  scales  or  very  small  at  a fixed  scale. 
Walter’s  result  [17]  for  the  covariance  is  a special  case  of  Proposition  2: 


CUM(Wjlkl,Whk,) 


o,  for  lii  -j2\  > 1, 

0(1*1  -fc2|-p),  for  ji  = j2. 


It  is  interesting  to  note  that  the  results  stated  in  Proposition  2 hold  for  any 
choice  of  F in  the  definition  of  the  Meyer- type  wavelet.  An  open  problem 
is  to  determine  if  there  exist  some  distributions  F which  will  significantly 
reduce  cumulants  between  wavelet  coefficients.  In  this  direction,  Zayed  and 
Walter  [18]  minimized  the  covariance  between  wavelet  coefficients  by  using  a 
bi-orthonormal  wavelet  basis  that  is  a function  of  the  original  covariance. 

In  Propositions  1 and  2,  different  expressions  of  the  wavelet  cumulants 
were  derived.  A natural  question  is  whether  or  not  the  cumulants  of  the 
original  process  can  be  expressed  in  terms  of  CUM(Wj1k1,  Wjmfcm)-  Thus, 
our  next  result  is  the  converse  of  Proposition  1. 
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Proposition  3.  Suppose  that  {Wjk}  is  a sequence  of  variables  with  all  Unite 
moments.  If  the  process 

CO  oo 

m=  E E 

j=— oo  k=—oc 

is  well  defined  (in  the  mean-square  sense),  then  we  have 

oo  oo  m 

CUM(X(t1),...,X(tm))=  E - E n i>uMCUM(Wjk). 

ji,k1=-oo  jm,kmz=- oon=l 


§5.  Conclusion  and  Future  Work 

In  this  paper,  different  relationships  between  wavelets  and  cumulants  have 
been  presented.  Results  show  that  the  wavelet  transform  is  not  only  a good 
tool  to  de-correlate  a Gaussian  process,  but  it  also  gives  small  higher-order 
cumulants  of  a non-Gaussian  signal.  The  Meyer-type  wavelet  is  particularly 
well-adapted  for  stationary  processes  since  they  give  null  wavelet  cumulants 
at  distant  scales. 

The  combination  of  wavelets  and  cumulants  has  not  yet  been  fully  ex- 
ploited. The  statistical  study  of  estimators  of  the  bispectrum  based  on 
wavelets  is  of  particular  interest  for  application  with  real  data  sets.  Also  in- 
vestigating the  properties  of  non-linear  and  non-stationary  times  series  models 
using  bi-spectral  methods  and  a wavelet  decomposition  approach  needs  further 
research. 

Acknowledgments.  This  work  was  supported  by  NSF  grants  DMS-9815344 
and  DMS-9312686. 
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